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INTRODUCTION 

This report on the impact of the Career Opportunities 
Program (COP) is divided into two sections, Volumes IV and V 
of the total Innovation and Change report- Volume IV con- 
tains the summarized results of the research efforts supported 
by this contract. The Appendix (Volume V) describes in 
detail the methodology of the study, including operati.onal 
procedures, instruments and field procedures. 
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I. OVERVIEW 

In June, 1971, the Office of Education, through its office 
of Planning, Budgeting and Evaluation (OPBE) contracted with 
Abt Associates Inc. to conduct an impact evaluation of the 
National Center for the Improvement of Educational Systems (NCIES) 
formerly known as the Bureau of Educational Personnel Develop- 
ment (BEPD) . The study, evolving from discussions between the 
contractor and the Office of Education, was composed of two 
sections— a series of project case studies and an analytic sur- 
vey of the impact of the Career Opportunities Program (COP). The 
present rt-oort addresses the survey of the impacts ot the Career 
Opportunities Program. 



1 . 0 The Career Opportunities Progr 



am 



The Career Opportunities Program is unique among federally- 
funded teacher training programs. Since its first year of opera- 
tion in 1969, it has endeavored to both provide a means of tea- 
cher training for many people who would not otherwise have en- 
tered the profession, and at rhe same time provide some benefit 
to the local school systems participating in this training pro- 
cess, cop's target population is, broadly stated, the disad- 
vantaged; low-income people, minorities, welfare mothers, veterans, 
parolees and other types of "high risk" individuals who would 
not normally seek emplo^TTient in education. m addition,- the 
schools in which participants are placed in practicum situations 
^re generally low-income schools. Title I schools, schools 
with h:igh enrollments of minority children and so on. 

The training procedures typical of COP projects differ 
from the traditional methods of teacher education in that they 
include (as a condition of the grant) extensive practicum ex- 
perience for the participants throughout their entire college 
career. COP participants arc employed as parapxof essionals 
("teacher aides") in the classrooms for a significant part of 
the day (usually averaging around thirty hours a weex) and also 



attend college courses leading to a Bachelor's degree in edu- 
cation and teaching certification. This practicum experience 
serves many purposes. As employinent , it serves to supplement 
the income of the participants who receive only tuition from 
project funds. In many cases, these earnings represent an 
aide's total income. Of course, the practicum activity in the 
classroom provides valuable interaction with children that can 
only make COP participants better teachers. And finally, a 
well-trained paraprof essional in the classroom serves the school 
by lightening the burden of regular teachers and thereby improv- 
ing the quality of education for the students , 

To summarize then, this program, which was nationally 
implemented in 1969, is unique in that it enables participants 
to be paraprof essionals /teacher aides) and go to college at 
the same time. It is expocted that the opportunity to go to 
college wJ 11 permit many of the participants to become full 
professionals in the teaching field. Since the program is tar- 
geted to lovzer- income participants and ethnic minorities, the 
hope is that this program will act as a vehicle for upward 
mobility. 

Since the COP aides are studying to become teachers and 
since they have been picked to reflect the ethnic background of 
the children they work with, it is further hoped that a bene- 
ficial impact on children can be shown. 

Finally, the COP presence on campuses, in classrooms and school 
systems in the state might be expected to exert some changes on 
these institutions. 

In the light of the above expectations, a study was de- 
signed which consisted of a survey of respondents including 
a two-wave study of selected behaviors of COP aides (i,e, , 
in the Fall of 1971, Spring of 1972). The purposes of this 
pre/post procedure were to investigate the* changes which 
occurred in the aides over time. 
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2. 0 Outline of the Study 

Based on the above goals of COP, v;e developed several 
very general areas of interest. We may summarize our study 
sfforts and concerns in the following very general hypotheses, 
stated positively: 

1, COP has been effective in enhancing the career po- 
tentials of its participants, 

2, COP has been an effective tool in improving the edu- 
cational experiences of students in the classroom, 

3, COP has engendered change in institutional structures. 

In order to investigate these hypotheses, several respon- 
dent groups were identified and surveyed. Wherever necessary, 
specialized instruments (questionnaires) were developed for each 
of the respondent groups. These groups were: 

• COP aides 

• Students in classrooms with COP aides (and an asso-- 
ciated comparison group) 

• Teachers of classrooms with COP aides (and an asso- 
ciated comparison group) 

• Principals of participating schools 

• Superintendents of participating LEA ' s (Local Edu- 
cation Agencies) 

• Administrators of participating IHE's (Institutions 
of Higher Education) 

• Project Directors 

• Certification Officers of the 50 State Education 
Agencies 

2.1 COP Aides 

A primary goal of COP is to enhance the career poten- 
tial of its participants. This is done in several ways. The 
primary intention of the program is to train teachers, and this 
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training, whi may lead to a college degree, is invaluable 
to the emp' otential of participants. But even if a 

participan lot complete the program and leaves before 

completion of the 4-year cJegree requirements, he may have 
earned an Associates degree and will in any case have had much 
valuable experience and training as a paraprof essional . 

In addressing the program's impact on these participants 
we were interested in the effects of their training on their 
grasp of basic skills, as well as on their career aspirations. 
Basic skill level was evaluated by two subtests of the AJult 
Bas ic Learning Examination (ABLE) The ABLE was used at the 
request of the client as a culture fair test appropriate for 
the aides. Career aspirations were addressed by secti ^ns of 
a questionnaire administered to participants- These measures 
were taken in both the Fall and Spring survey. In addition, the 
Fall questionnaire collected demographic information while the 
Spring questionnaire elicited attitudes and opinions of COP aides 
toward their experiences in the program. 

2 . 2 Students 

Impact on students was measured by achievement test scores 
and student opinions about school, self, and others. Standard- 
ized achievement test scores (reading and math subtests) sup- 
plied by the sampled sites were obtained for students (grades I-VI) 
with COP aides and comparison students who did not have COP aides. 
The question of interest here was whether children in classes 
with COP aides perform better on standardized tests than simi-- 
lar students who did not have COP aides. In one sampled site 
where achievement testing was not scheduled in the Spring, the 
evaluator administered the Cooperative Primary Tests (for 
grades I-III) and the California Test of Basic Skills (for 
grades IV-VI) , reading and mathematics subtests. 

Adult Basic Learning Examination (ABLE) , Mathematics and 
Heading Subtests, Harcourt, Brace and World, Inc. 757 Third 
Avenue, New York, New York. 



It is generally agreed that achievement gains are diffi- 
cult to measure. Furthermore, academic achievement alone has 

■en the total goal of public education. Hopefully, schoo 
-eii. nxldren achieve a strong and positive sense of self, othe 
and school. These opinions cannot be measured in an achieve- 
ment test. In order to assess opinions about self, others and 
school among students in grades I-VI, an instrument was 
developed called the "Student Opinionnaire . " It was administere 
to all students in the sample and comparison classes to 
probe for differences in attitudes toward school, self 
and others between children who have a COP aide in their class- 
room and those who do not. 

2 . 3 Institutions 

Institutional impacts were measured by exploring changes 
in classroom, elementary schools, local education agency (LEA) 
standards, institutions of higher education (IHE) courses and 
degree requirements and State professional certification require 
ments as described below: 

© Teachers in the student sample classes were asked 
to respond to a questionnaire which was designed to 
elicit information on instructional scheduling, extra 
classroom school-related activities, attitudes to- 
wards the Career Opportunities Program and selected 
demographic variables. Of prime interest is the re- 
lationship between presence of a COP aide and the 
amount of instructional and extra-classroom scho61- 
related activities reported. 

• Principals who have COP aides in their schools were 
asked to give their opinion on the impact of COP 
aides, both on classroom activities and classroom 
structure . 



• Superintendents of districts with COP aides were 
queried to obtain data on changes in district employ- 
mf^'^ ' policies, the nature of the district's career lat- 
tice (if any) and the history of upward mobility with- 
in the school system. In addition, attitudes toward 
COP aides were assessed. 

• College Administrators received an instrument that 
explored the relationships between the presence .f 
the COP project and changes in courses, admissions 
procedures, and degree requirements. 

3 Project Directors were administered a brief instru- 
ment to obtain information on recruitment procedures, 
community interest in services of COP aides, function 
and structure of advisory councils and some factual 
information for validation purposes. 

• The impact of COP on a specific institution, the 
State Education Agency (SEA) , was assessed by sur- 
veying the 50 state certification officers. They 
were asked the nature of recently instituted or 
planned changes in state certification requirements 
and if these reflected a concern for the needs of COP. 

3. 0 Selecting the Respondents: Sampling Procedures Used 
and their Implications for the 16 Site Survey 

COP Aides : The. evaluation and the impact of the COP 
experience on the teacher aides involved as participants was 
assessed through instruments administered to 480 aides in 16 COP 
sites throughout the country. Selection of the participants 
required a two-stage sampling. 
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Initially, 16 sites were selected from the 90 COP pro- 
jects funded in FY 1972. The great majority of COP projects 
have between 30 and 200 participants each. The exceptions are 
either so large as to swamp the analysis or so small as to 
have no appreciable effect on the results; we therefore restrict- 
ed the survey to projects within this size range. 

The 16 sites were selected at random from the 90 sites 
with the probability of selection proportional to project size 
as measured by number of participants. Within each of the 16 
sampled sites, 40 participants were chosen, 20* in their first 
year in COP and 20* in their second, for a total effective 
participant sample of 480. Selection was made by simple random 
sampling . 

Students : Due to constraints of time and budget, as 
well as the somewhat lower priority assigned to this aspect of 
the evaluation relative to the other sections, the student sample 
is somewhat less precise than those for participants or insti- 
tutions. This sampling plan was developed in consultation with 
representatives of the Office of Education , OPBE . 

The student sampling was a multi-stage sample/ selecting 
COP project sites at the first stage and classrooms with aides 
at the second. Intact classroom units were used; that is, all 
students in a selected classroom are therefore included in the 
sample. The first five sites selected in the 16 site sam- 
pling constituted the first stage of the student sample. Within 
each of these sites, a stratified sample of classrooms containing 
COP aides were selected. Two classes from grades I~VI were 
chosen for a total of 12 classrooms per site or GO over the 
five sites. 



★ 

To allow for expected attrition, 40 names were selected from 
each site with the expectation of analyzing data on 30. 
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We may fruitfully conceptualize the student testing part of \ 
the study as an experiment in v7hich there are groups which receive 
different treatments. This is a classic analysis of varian^^n r^. 
sign; i.e.; a nested group within treatment Ciu.. 
Our design may be characterized as a mixed-mode desian with 
subjects nested within classes nested within methods: 

"We begin by conceptualizing treatment population; differing 
systematically only with respect to the level of treat- 
ment variable, A. From each population, subjects are 
randomly selected in groups ot size n; the sampling 
process ends when g groups of n subjects have been sam- 
pled from each population.'** 

In this model; the total variability among subjects is composed 

of three sources: 

• Treatment Effects: COP aide/no GOP aide 

• Group Effects: grade/class 

• Residual Individual Effects: attitude/ability 

The paradigm below defines the design where, 

= Treatments (fixed variable) 
1 A^ = COP aide in class 

A^ = No COP aide in class 




Groups (random variable ) 



Classes chosen within constraint of two from 
each grade/site 

= Subjects 

Students within each classroom 



* Myers; J.L., Fundamentals of Experimental Design , Allyn and 
Bacon, 1966; p. 213. 
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^1 s 
^1 ^1 



^2 ^2 



'60 



^1 ^2 



^1 ^1 
^2 ^2 



60 



^30 ^30 



30 



^30 ^30 



N 



30 



The sources of variance and degrees of freedom are described 
in the table below: 

Analysis of Variance for Groups-within-treatments Design 

SV df df 

Total agn-1 3599 

Between G ag-1 119 

A a-1 1 

G/A a (g-1) 118 

S/G/A ag(n-l) 3480 



The sample frar^e was developed in conjunction with the 
Project Director of e.Lch site and, usually, an LEA representative. 
He identified all clas srooms in his site that had COP aides in 
them. From this list a sample of 12 classrooms, two each from 
each of the first six -^rades, for a total of 60 classrooms was 
selected. 

A sample of ccmparison students was also drawn from each 
of these LEA's. A frame consisting of classrooms of grades I-VI 
without aides in schools that, in the opinion of the evaluation 
staff of the particular LEA, are -omparable to COP schools in 
socio-economic status cr-aracteristics was developed. 

Institu • onal ? esoondents ; The sample* of teachers was se- 
lected at the Seine tine as the sar^ple of classrooms. Each 
classroom unit '^^ lected had a t ..ler associated with it. The 
teachers from t classroorr^^ se^ - ' ted above constitute the teacher 
sample. Precis n of generaliza-. is will be the same as for 
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the classrooms. Contact will be established in the same manner as 
for classrooms. A comparison group of teachers was drawn 
from the comparison classrooms in the same way. 

The selection of 16 COP project sites as the first stage 
of the participant sample determined the sample of. Principals 
of participating schools, Superintendents of participating 
lea's, IHE Administrators, and COP Project Directors. All of 
these individuals in the 16 sites already selected were included 
in our survey. In cases where there was more than one indi- 
vidual of a given type (as when there are several participating 
IHE's), we include representation from all of the appropriate in- 
stitutions. Generalizations from data so gathered to the restricted 
universe of 90 COP projects have the same precision as the first 
stage of the participant sample. Selection of all possible" respon- 
dents at a selected site (100% subsampling) adds no additional error 
to the estimate. 

The SEA survey represented a census in that 100% sam- 
pling was employed. 

The Three-Site Survey 

On September 29, 1971, at the request of the client, three 
additional sites were added to the survey in order to gain data 
from sites with more than 200 participants. These sites had 
been excluded from the original sample frame. The three sites 
were Chicago, Illinois; Detroit, Michigan; and Bedf ord-Stuyvesant 
(Brooklyn) , New York. Since these sites were selected judgmentally 
and not in a probabilistic manner, we could not discuss precision 
of generalizations if the data from these sites were combined 
with the data from the other 16 sites. Consequently, while the 
participants were administered the same instruments as the main 
sample, their data was tabulated and will be discussed separately. 
Students and teachers from these sites were not included in 
the survey. Findings for the three-site survey are described in 
the Appendix to this volume. 
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II. HIGHLIGHTS OF FINDINGS 

1.0 COP Aides 

The COP Project Director's Handbook describes how parti- 
cipants in the program should be selected. Preference should 
be given to: 

a. (Primarily) low- income people 

b. Ethnic mix that reflects the community served 
Veterans 

d. People at the high school or equivalent levels 

e. Students participating in Youth-Tutoring-Youth 

f. Males 

g. High risk potential 

Don Davies, former Director of MCIES, described the COP program 
as follows: 

"It offers a chance to bring into the schools persons 
from low-income areas who would not normally enter col- 
lege, and who normally would not consider teaching as 
a Career . " 

WHO ARE THE COP AIDES? 

Overall , our survey shows that COP aides tend to be: 

9 from familie whose present total yearly income xs 
under $6000 (51%) 

© drawn from ethnic minority groups (68%) 

• female (74%)* 

• young (69% are under 35 years) 

• high school graduates or more (93%) 

• drawn from other aide programs (46%) 

• presently living in the community where they are an . 
aide (53%) 

However, when we separate out first and second year participants 
m tne program, a number of significant shifts are evidenced. 




While only 26% of the total sample are males, almost one-third 
of these (30%) are veterans. 
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Table I presents distributions of pertinent demographics for 

the total sample of COP aides. As compared with participants , 

who have been COP aides for two years, the newer recruits (first 

year COP aides) show shifts towards: 

• more males (from 18% to 39%) 

a better educated (from 42% to 54% have had some college 
training before entering COP 

• less minority ethnic representation (from 72% to 62%) 

• younger (from 45% to 68% are under 30 years) 

While the increase in males is to be applauded, the re- 
duction in minority representation and the increase in prior 
college training may be viewed with some concern. Does this 
shift follow some self-selection principle or does it reflect 
a shift in recruitment policies at the local level? 

One LEA administrator summarized a prevalent; attitude 
when he saia, "The time has come to shift the emphasis in 
teacher education from quantity to quality," by which he meant 
that cop's recruiting policy in its first year was perceived 
as favoring "high risk" participants and that future parti- 
cipants should look more like the traditional teacher. 

Alternatively, there is some data to suggest that the 
supply of readily available high risk candidates was signifi- 
cantly depleted in the firs' year's recruiting. 

• 58% of the second year participants were teacner 
aides before they joined COP, as compared with 41% 
of the first year. 

• 44% of the first year participants first heard of 
COP from a friend (and hence can be considered essen- 
tially self-selected) , while only 30% of second 

year students followed this path. 

It would appear that recruitment may have been more ac- 
tive at the beginning of the COP program than more recently. 
39% of the second year aides heard of COP via the LEA or a ] 
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TABLE I 

SELECTED DEMOGRAPHICS OF COP AIDES 
FOR TOTAL SAMPLE, FIRST YEAR, AND SECOND YEAR AIDES 



Total 
COP Aides 

(540) 

% 

26 
74 



COP Year 
First Second 

(213) (323) 

% % 

39 18 
61 82 



Sex 

Male 
Female 

Education Completed 
Grades 4-6 
Grades 7-8 
Grades 9-11 
Grade 12 
Some College or more 

*less than .5% 

Age 

Under 20 

20-24 

25-29 

30-34 

35-39 

40+ 



1 0 1 

0* 0 1 

6 5 7 

46 41 49 

47 54 42 

6 7 6 

28 38 21 

20 23 18 

15 12 18 

13 8 16 

18 13 22 
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Ethnic Identification 
American Indian 
Black 
Oriental 
Mexican 
Puerto Rican 
Other Spanish 
Other V7hite 
Other 



Income 



Under $1000 
$1000 to 1999 
$2000 to 3999 
$4000 to 5999 
$6000 to 7999 
$8000 to 9999 
$10000 to 11999 
$12000 and over 
No response 



Total 
C OP Aides 

(540) 
% 



COP Year 



First 
(213) 



S econd 
(323) 



6 

45 
1 

13 
2 
1 

31 
1 



6 
40 
3 
8 
1 
2 
36 
2 



6 
49 

0 
16 

1 

0 
27 

1 



(435) ** 
2 

4 

•» 

22 
23 
14 
12 
11 

7 

5 



(161)** (275)** 



2 
6 
24 
23 
17 
11 
6 
7 
4 



2 
3 
21 
23 
13 
12 
13 
7 
6 



* 

Change in base represents attrition between Fall and Spring. 
Income question asked in Spring. 
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government program as compared with 23% of the first year aides. 
It is a tribute to the sucr- : of COP that it became well known 
enough to reduce the nee y of very active recruiting; however, 

high ;isk candidates usua require outreach methods to bring 
them in. 

In order to tease out possible explanations for the shift 
in profile between second and first year COP participants, Pro- 
ject Directors were questioned about their recruitment practices. 
Eight Project Directors indicated that they had changed their 
recruitment practices in the second year. When asked to specify 
what these changes were (open ended question) their responses 
were of three kinds: 

1. Termination of active recruitment 

2. Selection of only the "best" candidates 

3. Selection of candidates by advisory council 

The Project Directors were also asked the following 
open-ended question : 

"What are your present recruitment procedures for 
COP aides?" 

The eight statements below are responses to this 
question by the eight directors who subsequently indicated 
(in a later question) that their recruitment policies had 
changed. 

1. "None necessary anymore — excellent applicants waiting 
in line for attrition to occur. Word of mouth at 
project schools is all, now." 

2. "We now only work toward replacement of withdrawal 
vacancies looking for the best possible participant 
prospects . " 

3. "Much more sophisticated. Much more valid (opinion). 
More time . " 
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4. 



"Veterans first priority. Aides with previous college 
credits due to phase-out COP by 1975." (They are) " screened 
by entire advisory committee and by committee oi: three." 



5. "Filling vacancies only. No recruitment. Applications — 
testing--interview. " 

6. Application — Interview — Advisory Committee selection • 

7. "This year used field staff and tribal councils. 

8. "YTY (Youth Tutoring Youth) experience for new bi- 
lingual aides earlier identification of YTY candidates." 

WHAT ARE THE CAREER PLANS OF COP AIDES? 

The aides v/ere asked what career plans they had for the 
next year and the next three years. The questions were closed 
ended; that is, a list of alternatives were presented and the 
aide checked the answer or answers closest to his, or her career 
plans. A fill-in space was provided if none of the options 
were appropriate. To measure degree of indecision, respondents 
were informed that they could check more than one box. Thus, 
multiple responses can be viewed as an index of indecision 
regarding future career plans. 

The major findings .on career plans for COP aides follow: 

» There is very little indecision on career plans for 
the nG::t year, i.e., 15%. When plans are for the 
next three years, indecision increases to 30%. 

• Most (8 9%) intend to continue in the COP program 
next year. 



• More than half (59%) plan to become teachers within 
the next three years. 

• Surprisingly, 46% plan to be in the program three 
years from now. This includes 40% for whom this is 
the second year in COP. Since a goal of COP is 

to encourage attainment of a college degree, 
this suggests either that the participants 
misunderstood the question or they are not properly 
informed as to their responsibilities and priviledges 
in the Career Opportunities Program. 

The percentages quoted above were taken from responses 
to the Fall survey. m the Spring survey, responses were vir- 
tually the same. Percentages varied by no more than one or two 
points; there were no significant differences in the group re- 
sponse in the spring compared with those of the fall. 

TO explore the impact of COP on their employment plans, 
aides were asked the following open-ended question: "What 
do you think you would be doing this year if COP did not exist?" 
Multiple responses were permitted and a code was developed to 
reflect the variety of responses made. The most frequent response, 
made by 21% of the total aides, was that they would work in a 
classroom as a teacher or other education professional or para- 
professional. When, however, the aides are separated into male 
and female, there is a significant difference* in the response; 
that is, the distribution is as follows: 

Males 5% 
Females 26% 

Furthermore, 25% of the males as compared with 15% of the 

females inaicatea a possibility of working in a non-teacning field. 

on the ocher hand, more males than females stated that 
chey would become students (27% vs. 15%) or would combine 
school and work (25% vs. 10%) . 
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What seems likely is that if COP were discontinued there 
would be a significant eduction in male teacher aides. Given 
the importance of a mal presence in the - lassroom not only to 
help low-income males, out more important,/ to act as role mo- 
dels for the children, such an eventuality would be most unfortu- 
nate. 

WHAT. PERFORMANCE DIFFERENCES ARE THERE BETWEEN MALE AND FEMALE 
COP AIDES? 

The male COP aide appears to be more actively pursuing the 
career goals implicit in COP than the female aide. This is 
evidenced as follows: 

Compared with the female, the male COP aide: 

• Takes more college courses and spends more hours 
in college classes 

• Spends more time on college work doina homework and/or 
reading for courses 

• Scores higher on both reading and math sections of 
the ABLE test 

• Attends more neighborhood activities 

• Is more likely to rate himself "Better" than most 
other aides on a variety of teaching performance/ 
competence measures. 

Tables II and III which follow present ' data - on these selected 
measures . 



1 
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TABLE II 



COMPARISON OF MALE ANJ FEMALE COP AIDES ON SELECTED MEASURES 

TOTAL MALE FEMALE 

(435)* (104)* (331)* 

% % % 



Number of university courses 

presently taking 

1-4 55 20 65 

5 or more 30 57 22 

no answer 15 23 13 

Number of hours a week spent . 

attending college classes 

less than 12 68 35 79 

12 or more 30 65 18 

no answer .2 0 3 

Number of hours a week spent 

doing homework or reading for 

college courses 

less than 12 65 48 71 

12 or more 32 51 25 

no answer 3 14 

Scores on ABLE Reading Test* * (Spring) 

Less than 42 51 31 57 

42 or more 49 69 43 

Scores on ABLE Math Test^ * (Spring) 

Less than 30 70 46 78' 

30 or more 30 54 22 

Number of school neighborhood 

activities attended 

1-2 " 19 24 17 

more than 2 27 32 26 

none or no answer 54 44 57 



Weighted Bases 

ABLE Reading Test has maximum score of 60 
ABLE Math Test has maximum score of 42 
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TABLE IT I 



differ: :es ErrwEEN .male and female performance self-ratings 
"AS co' c \red '•■:th most other aides" 



TOTAL 
(433) 

% 



Male 
(104) 



female 

(331) 



Percentage rating self as 
"Better than most other aides" 
on : 



knowledge of subject matter 


21 


33 


^ * 


17 


presenting lessons to class 


23 


33 


>* 


20 


getting ideas across 


25 


40 


>* 


20 


getting along with parents 


27 


22 




28 


understanding needy of neighbor- 
hood children 


31 


35 


> 


30 


getting along with children 


41 


54 


>* 


37 


ability to instruct children 
individually 


31 


49 


>* 


25 


ability to tutor 


25 


39 


>* 


21 


ability to instruct children 
in small groups 


27 


31 


> 


25 


lesson planning 


20 


23 


> 


19 


college performance 


20 


24 


> 


19 



Diffferences are significant at p4..01 
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WHAT IS THE COP EXPERIENCE? 

How do COP aides feel about their experiences in the 
program? 

When asked to select among five measures ranging from 
very unsatisfying to very satisfying, 97% stated that they found 
the experience as somewhat or very satisfying ("very satis- 
fying," accounted for 68%). This is a very strong vote 
of confidence in the program! 

In order to explore attitudes further, we asked two 
open-ended questions. They were as follows: 

"What is it about the program you like the most?" and 

"What is it about the program you like the least?" 

such open-ended questions frequently raveal attitudP^ 
.. that a closed ended question would be unable to elicit. The 
results indicate a very strong, positive attitude towards the 
program. 

First, the number of responses to "like the most" was 
much greater than those to "like the least" i.e., 623 vs. 323 
(this latter figure does not include 85 responses which stated 
they had no. complaints) . 

Secondly, while the first question had only a 3% non- 
response rate, the second had 17%. 

Finally, while there was no single complaint that was 
made by more than 9% of the respondents; by contrast there was 
strong agreement by respondents on what they liked the most, 
i.e., the opportunity for an education (61%). 

The four most frequently cited items in each category 
were as follows: 



"Like the most" 

Chance to get an education (61%) 
on the job training (23%) 
financial assistance (19%) 
giving service (11%) 
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"Like the least" 

no complaints (20%) 

not enough money (9%) 

schedule too full (9%) 

time classes are held (7%) 

communication problems with 
the administration (7%) 
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The opportunity for an education is, of course, a heavy 
component of the COP program. its importance is reflected in 
the aides citing the "opportunity to get an education" as what 
they most like about COP. What are the components of this 
educational experience? In general, courses are required rather 
than chosen by the aides. 

The majority of COP aides 

© Take 3-5 courses a semester (63%) 

• Spend less than eleven hours a week on homework 
or reading for courses (65%) 

• Rate their courses as "about right for me" (90%) 

HOW DO COP AIDES FEEL ABOUT THEMSELVES AS PROFESSIONALS? 

In order to assess the professional self-image of 
the COP aides we asked them three questions. We asked them to 
rate themselves on specified performances as compared to other 
aides; we asked them to rate themselves on specified performances 
compared with their supervising teacher; and finally, we asked 
them to rate the frequency of finding it a problem to carry out 
specified activities. The results of these appear in Tables 
IV and V which follow. 















Q) 
















O 


u 
































0 c 






cn 








c 
















w 


w 






•H 








> 










-P 0 






0) 


0 0 






a 


IS tJ^ 










Q 




CO 










0) 


< 




jC 


g 














•H 




CO 








w 






0 Q) 


Q 








H 




(1) 


< -P 






U 




•K 


m 






LD 




1 


W 


CO 


H 


■P U 


n: 




0 


Q) <D ( 




11 


u 




O 









CW3 



00 



LD 



00 



LD 



in 



o 



00 



00 
LD 



tN 



O O 



IN H 



o 

LD 



W 

< 
Eh 



E-i H 
CO 
S H 

CO w 
Q D 

M CO 

CU Eh 
O H 

u :2 

Ci^ Q 
O W 

z a, 

H s 
Eh O 



ERIC 



CO m 






O CD 




u 




0) 




^ '"^ 


id 




Eh 


m 


0 C 


H Cm 






12 O 


u 


U 


<\) 




Q 


x: 


w 


W H 




m 




0 








-P 0 


Oh 0 


4J 


0 0 




w 




O 


0 








= C 






0 


jC 


•~; 


W E 


4J 


m 


u m 


•H 


-p tn 






0 Q) 




'd 








< -P 




u 


&^ w 


m 






a 


1 


w u 


e 


-P U 


< 


0 




w 


u 




D:: Eh 







O 

H 

(1) 
CO 

MH 
O 

c 

0) 

m 
c 
o 

M 

Q) 



CN 



U 

0) 

-p 
-p 

e 
-p 

o 

0) 
•o 

O 
O 

H 

o 
c 



CN 



m 

H 

o 

o 
-p 

to 
c 
o 

U) 



tn 
C 
•H 

-P' 
C 
0) 

U) 



LD 



in 

CN 



W 

to 

O 

u 
o 
m 

to 
m 

0) 
•H 

tn 
C 
•H 

JJ 
-P 
Q) 
O 



cn 



CN 





o 


CN 






»^ 




»^ 




CN 


H 


in 


CN 


ro 


ro 


rsj 




1 

rH 








•H 








jC 








O 


C 








0) 




to 


M-t 






4J 


0 


•d 




C 


0 


H 




Q) 


■ to 0 


•H 














a' u 


o 




a 


Q) 0 










-p 




x: 


x: 


0 




-p 




-3 




•H 




U 


U 






4J 


0 




c 


to 


-P 


tr 




C 




C 


C to 


•H >i 


-P 


o 


•H -H 


H 




H 


^d x: 


0 rH 




03 


C -P 


-p m 


-p 




m 






tn 


-P MH 


>iTl 


>1 


C 


to 0 


4J -H 


4J 


•H 




•H > 


•H 




0) c 


H -H 


H 


-P 


'd (D 


'H 


•H 


(D 


c u 




J3 


a 




<:.H 





CN 



'd 

H 
•H 

x: 
o 

-p 
o 

-p to 

to Qu 
C 3 
•H O 
U 

O tn 



-P (U 

H to 
•H 

<J -H 



o 

CN 



tn 
C 
•H 

m 
a 
c 

0 

to 
to 

0) 



0) 
-P 

x: 

tn 

•H 

(1) 



TABLE V 



ERIC 



FREQUENCY OF FINDING ACTIVITY A PROBLEM AMONG TOTAL COP AIDEt 

(N=435*) 



Very Ere- occa- . sel- very no 
freq . quent sionally dom seldom answer 



Activity 



Weighted 



Giving a class lesson 2 2 10 33 43 10 

Getting children to 2 2 7 29 52 8 

work with me 

Getting along with 3 2 5 13 67 10 

supervis ing teacher 

Maintaining discipline 3 7 34 25 22 9 

in class 

Getting children to 2 — 5 19 55 9 

like me 
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COP aides appear to have a good professional self-image. 
This is evidenced in several v;ays. 

First, few describe frequent problems in any of the ac- 
tivity areas listed. The only one that is seen as an occasional 
problem by a fair percentage is maintaining discipline in class 
(44% see this as a problem occasionally or more frequently.) 

Secorxd, they perceive their performance as at least as 
good or better than most other aides. This very positive self 
image is somewhat tempered when the aides compare themselves 
with teachers, but even here, with the exception of "lesson 
planning" and "knowledge of subject matter" the majority of aides 
see themselves as at least "about the same" as their supervising 
teacher. 

It is instructive to note that 34% of the aides perceive 
themselves as better than their teachers in "understanding the 
needs of children of this neighborhood." Prior to conducting 
the impact study and also during several site visits, teachers we 
spoke to frequently mentioned how helpful the aides are in ex- 
plaining the children and their special customs and needs to them. 
The teachers as a rule do not live in the neighborhood of the 
children; they therefore cannot be as aware of the current status 
of the child. As an example, a teacher in the Worchester, Massachu- 
setts COP program (which we visited for orientation to the program) 
stated that her COP aide was invaluable because, among other thinqs, 
she can warn her in advance when one of the children has had a family 
crisis so that if the child seems distracted or ornery, the teacher 
will recognize the cause and not misunderstand the child's motives. 
This teacher also stated that she never knew until her aide ex- 
plained it to her that certain children greatly dislike being 
patted on the head. The teacher replaced this gesture of affec- 
tion with one that was more appropriate for these children. 

In bilingual classes, the teacher rarely can speak any 
language but English; the bilingual aide is indispensable here 
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in being able to communicate with children who have an English 
language problem. This is particularly so in the primary grades 
where so many attitudes towards school and learning become fixed. 

In consideration of this, the 34% who see themselves as 
better than their supervising teacher in this area are paralleling 
the teacher's perceptions as described above. These impressions 
were subsequently confirmed by the results of the teacher's sur- 
vey. The teacher sample was selected from the five sites which 
included student testing and is therefore more limited than the 
COP aide sample. However, it was chosen independently of the aide 
and can be viewed as a validating sample for these impressions 
of COP performance. Table VI describes the ratings given by teach 
ers with COP aides on performance measures. When comparing their 
aides with an average teacher, 53% of the thirty teachers sampled 
saw their aides as oetter than the average teacher in "under- 
standing the needs of neighborhood children." 
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DO COP AIDES IMPROVE IN TESTS OF BASIC SKILLS DURING THEIR 
COLLEGE TRAINING? 

The Adult Basic Learning Examination was given to the 
sample of first and second year COP aides in the Fall of 1971 
(Level III, Form A) and the Spring of 1972 (Level III, Form B) . 
Although national norms do not yet exist for these tests, they 
were selected at the request of the client since they came closer 
to a culture free test for adults than other more traditional 
tests. 

Our research interest focused on differences in performance 
between COP aides who had been' in the program for one year com- 
pared with those who had been in the program for two years. We 
expected to find a significant difference favoring second year 
aides in the fall testing, and a reduction of this difference 
in che spring testing. 

The findings did not support the expectations. Indeed, 
there was a difference in performance between first and second 
year aides, but in a direction opposite from that expected; i.e., 
first year aides scored consistently higher than second year 
aides. 

The table below describes these findings. 



COP Year 



TABLE VII 

MEANS AND STANDARD DEVIATIONS OF FALL AND SPRING 
ABLE TEST SCORES FOR READING AND MATH AMONG FIRST 
AND SECOND YEAR COP AIDES 



Reading Math 

Fall Spring Fall Spring 

X SD X SD X SD X SD 



First 43.6 10.4 41.7 10.2 25*8 7.7 26.3 8.1 

(N=159) (N=160) (N=160) (N=160) 

Second 40.8 9.6 39.8 9.2 23.9 7.7 24.3 7.7 

(N=24 0) (N=2 39) (N=2 39) (N=239) 
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As has been documented in previous sections, there 
are major differences in the demographic characteristics of 
participants in the two groups. These differences are in 
areas frequently suspected of being related to achievement on 
standardi :^ed tests,. 

The relationship between subjects' scores on the fall 
and spring tests was very high. Fall and spring scores for 
reading achieved a correlation of .813; the correlation for 
fall and spring math scores was .797. However, in order to 
investigate the effects of various demographic variables on the 
performance of COP aides, a multiple regression analysis was 
performed, which incorporated both selected demographics and 
fall test scores as independent variables. This allowed us 
to investigate the contribution of each variable independent 
of all the others, statistically controlling for their effects. 
Spring reading and math scores were dependent variables pre- 
dicted by the following independent variables: 

• Fall test score (of the same subject 

• Year in program 

• Sex 

• Race (a dichotomy: white vs. nonwhite) 

• Previous education (dichotomized to: less than high 
school diploma vs. diploma or more) 

• Linguistic status (Dichotomized to: those who 

are native English speakers vs. those whose native 
language is something else) 

The analysis was conducted following the procedures suggested 
by Cohen, which are described elsewhere in this report. Full 
sets of data on all variables were available for 397 participants. 

Slightly different results were obtained for Reading 
and Mathematics. In Reading, a multiple correlation of .81726 
was obtained using all the above variables. For Mathematics, 
this value was .80674. This indicates that, respectively. 
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approximately 67% and 65% of the variance in the spring test 
scores could be explained by variance in the set of predictor 
variables. In both cases, the most important predictor was 
the fall testing result . 

One further analysis was performed to explore the con- 
tribution of each of the five independent variables controlling 
for the fall test scores, i.e., fall test scores act as covariates. 
What is of interest is to observe if, when all other significant 
factors are partialed out, year in the program accounts for any 
difference in performance. 

The following table presents the results of testing 
the significance of the independent contribution of each of 
the five independent variables. 



TABLE VIII 







Reading 






Math 








F 


B 




F 


B 


Year 


. 00028 


. 32798 


. 01734 


. 00001 


. 01114 


-. 00327 


Sex 


. 00417 


4 . 87161* 


-.06990 


.01338 


14 . 90669** 


-.12482 


Race 


. 00156 


1. 82743 


. 04688 


. 00126 


1. 40377 


. 04371 


Equc . 


. 00061 


. 71454 


-.02520 


. 00013 


.20054 


. 01362 


Lang • 


.00003 


. 03514 


-.00620 


. 00002 


. 02228 


. 00520 



*Pr. .05 
**Pr. .001 

The two most important variables are sex and race, 
effects which have been noted for many years. The direction 
of the regression weight indicates that men score higher 
than women on both tests. It is interesting to note that the 
importance of a participant's race is not statistically signi- 
ficant, although it is in the expected direction. The effect of 
the participant's year in the program, independent of his sex, 
race, previous education and linguistic history, is virtually zero. 

^ A r2 is the change in the multiple correlation resulting when 
the indicated variable is dropped from the regression equation. 
F is the F-ratio, with [1,389] degrees of freedom, resulting 
from the significance test of this change in r2 , and B is the 
standardized regression coefficient taken from the regression 
equation for the indicated variables. 
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WHAT DIFFERENCES ARE THERE BETWEEN COP AIDES WHO PERFORM WELL 
OR POORLY ON THE ABLE? 

When the ABLE was given in the fall, we expected to find 
difference-s between first and second year COP aides. We reasoned 
that participants who had already had a year of college in the 
program would perform better than those who were just beginning. 
We based this prediction on the assumption that there would be 
no significant demographic differences betv/een first and second 
year COP participants. As already described above, this assumption 
proved false; first and second year COP aiJ.^s have very different 
profiles with the more recent recruits having characteristics that 
would favor better performance and therefore swamp any differences 
which might have occurred between first and second year aides. 
When the demographic differences were controlled for, no differences 
between performance of first and second year COP aides could be- 
observed. 

Are there any distinguishing characteristics of aides who 
perform particularly well or poorly on the ABLE? That is, 
within the group as a whole, are there characteristics which 
differentiate performance? 

We know that the relative ranks of performers did not 
appreciably change since fall testing. Because of this stability 
we used the fall ABLE results to explore differentiating charac- 
teristics between high and low scorers on the ABLE. Table IX 
delineates our findings. 
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TABLE IX 



Selected Characteristics Differentiating High 
And Low Scorers on the ABLE Reading and Math Subtests. 
Fall Test Results 

A BLE 

Math Quartile 



Total 
Sample 
(540)-- 
% 



26 
74 



Reading Quartile 
Lowest Highest 



SEX 
Males 
F emales 

EDUCATION 
Less than college 53 
Some college or more 47 

ETHNICITY 
Minority 69 
Other 31 

AGE 

Under 24 34 
25 or older 66 

YEAR IN PROGRAM 

First 40 
Second 60 



■ % 



13 

87 

58 
42 

93 

7 

24 
76 

34*** 
64 



(139) 
% 



45 
55 

34 
66 

43 
57 

•45 
55 

51 
49 



Lowest 
(133) 

% 



13 
87 

63 
37 

94 
6 

21 
79 

31*** 

67 



Highest 
(144) 

% 



39 
61 

39 
61 

.40 

60 

41 

59 

44 
56 



p^.Ol 

\V eighted bases 

Sums to less than 100% to reflect three nonr espondents on Question 2, 
"Is this your first or second year in the COP program? " 



A further analysis was performed on ABLE scores control- 
ling for effect of education among ethnic groups. The results 
do not show any reduction of differences as reported in the pre- 
ceding table. 

These data must be interpreted with .caution. While it is 
dangerous to go beyond the data, a number of issues leap to mind. 
These are issues whose relevance to the data and to COP cannot be 
denied : 

• Are achievement tests appropriate to this evaluation 
or would performance based criteria be more useful? 

• Is the ABLE or any test culture free? 

• Minority groups are considered to be test weary; are 
they test wise? (25% of the aides scored below 
chance on the math section oi the ABLE suggesting that 
they did not guess, i.e., may have left answers blank.) 

• Is there a differential amount of test anxiety for the 
two groups? 

• What differences in quality of education exist between 
the earlier educational experiences of the two groups? 



These are not questions easily answered by either this 
study or the present state of knowledge about achievement testing. 
Until they are, ethnic differences as we have found cannot be 
interpreted . 

In the Spring survey of COP aides, an alternate form of 
the ABLE tests was given again. There were few shifts in 
ranks of the aides from those that they had fallen into in the Fall. 
For example, 70% of those who scored in the highest quarter 
in the math subset remained in that quarter for the Spring. The' 
remainder of them drifted down as would be expected due to regression 
towards the mean, but only one quarter so that 97% remained in the 
upper half. The results in reading were virtually the same as 
for math, i.e., 71% and 93%. 
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The lo>rest quarter showed the same tendencies. In math 61% re- 
mained in the lowest quarter. What movement there was was re- 
stricted to the bottom half and essentially the same pattern was 
repeated in reading. In other words, there were no dramatic 
changes in the relative scores or rankings. Most people who scored 
at the high and low ends of the continuum remained in the upper 
or lower half of the distr ibut ion • 

However, a further question remained. Is the score on 
the ABLE related to any of the performance/attitude measures we 
took in the Spring survey? To explore this, we compared per- 
formance/attitudes of COP aides who had scored in the highest 
quarter with performance/attitudes of those who had scored in the 
lowest quarter* We used the Vail ABLE to establish these groups. 

One noticeable difference is the much higher proportion 
of non-responses among those who score in the low end of the scale 
on the ABLE. These went as high as 80% on one question. The 
proportion of non.-response over all the questions was consistently 
higher amojig the low .scorers as compared with the high scorers. 

T ::d^e the results must be interpreted wi.:^h caution, it 
appears t.zH.'± the poorer performers on the ABLE test have difficulty 
with the crollege requirements as evidenced by the following? 

• ■;-l4% report that they found the college courses 

zoo difficult" as compared with 2-4% of the highest 
quart ile . 

• They devote less time to their college work (42-47% 
spend less than six hours per week attending classes 
as compared with 12-15% for highest quartile) . 

• They report fewer hours devoted to doing homework 
or reading for college courses, i.e., 51-53% report 
spending less than six hours a week on their work. 
Only 22% in the upper quartile report this. 



2.0 Students 



2.1 Student Achievement Testing Summary Findings 

One goal of the Career Opportunities Program is the im- 
provement of the learning experiences of children in classrooms 
served by COP aides. There are several possible indicators of the 
impact of COP aides; achievement test scores, attitude scale 
scores, and descriptions of the instructional processes of the 
teacher and aide in the classroom. All of the above measurements 
have been made in this study, although the present section is 
concerned only with the f irst— achievement test scores. 

Measures of reading and mathematics achievement were to be 
collected of a sample of students who are enrolled in grades I-VI 
in classrooms having COP aides as well as from a comparison group. 
These students were selected randomly from within five of the sites 
involved in the participant survey. Procedurally, these five sites 
were the first five selected when the participant sample was drawn. 
Comparison classrooms from within the districts involved were 
identified by the districts themselves with the intention of making 
the classes maximally comparable. 

Certain procedural constraints precluded Abt Associates staff 
from personally collecting these data in all cases but one. Conse- 
quently, the participating districts were asked to supply us with resu 
of their regular testing procedures for the sampled students as 
well as with related demographic information. A covariate was al- 
so requested to allow at least partial statistical control for 
initial differences in test scores between the COP -sample and 
their comparison group. Whenever possible, a previously administered 
reading or mathematics test was preferred. If such scores were not 
available, IQ scores were accepted. 

The data were analyzed by means of multiple regression 
analysis. This multivariate procedure was selected because it 
does not have the stringent experimental design restrictions of 
classical analysis of variance, and for the ease in which 
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covariates (test scores, demographics, etc.) may be handled. A 

variance partitioning approach of the sort proposed by Cohen (1968)* 

2 

was adopted. This procedure tests the increment in R , the 
squared multiple correlation obtained from the predictor variables, 
that can be had by introducing another variable into the prediction 
system. In this case, that variable was whether or not the student 
had a COP aide in his classroom. 

Table XI summarizes the results of a series of regression 
analyses. Due to the necessity of collecting data from the records 
of several school districts, the same set of variables was not 
available for all students. (See Table X which follows) Also, 
the different diLStricts used different tests and often used non- 
comparable tests in different grades within the district. Conse- 
quently, we wer'^ compelled to compute a series of analyses to assess 
the effect of C jP within each of the site-grade combinations. 



Jacob Cohen, "Multiple Regression as a General Data-Analytic 
System," Psychological Bulletin , 1968, Vol.LXX, No. VI, pp. 426-443. 
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It is possible to use nonparametric procedures to combine 
the results of these various tests. Binomial tests were applied 
to the regression coefficients obtained to det-nci-ne if their true 
value is different from zero. The results obtained are shown 
in Table XI. 

The overall result is that there is no statistical ly signifi- 
cant ev idence that COP aides have had an effecir on the achievement 
te st scores of students in their classrooms , ^his is not to say 
that COP has had no effect cn other aspects c± the educarional 
environment or on cognitive sl^ills not mea^sured by these tests 
or on affective aspects of -_.:.ie students. Indeed, the Student 
Opinionnaire instrument wras constructed to explore affective 
responses of students: A suininary of these findings follows. 
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TABLE XI 

STUDENT ACHIEVEMENT TESTING SUMMARY TJ.EI,:: 







M 


KTj AUXIN Li 


D ^ ^ 


MATH* 


D 


1 

J. 


1 


DD 


Q/1 /I 


OCT 

~ . 2d1 


9. 494 


C "5 O 

538 






J U 


, UUj 


. 02b 


.388 


+ . 308 




3 


42 


.048 


G57 


0 


0 


£. 




Q Q 
OO 


. UUl 


+ .039 


.001 


+ .030 


3 


4 


70 


.921 


-.265 








5 


76 


.153 


+ .064 






4 


1 


63 


.008 


+ .023 


4.062 


+ .444 




2 


67 


1.601 


+ .276 


1.472 


+ .253 




3 


46 


.066 


-.071 


1.126 


-.256 




4 


38 


.159 


-.093 


.121 


+ .099 


5 


1 


56 


.015 


-.030 


.370 


+ .172 




2 


65 


.170 


+ .088 


1.011 


+ .210 




3 


44 


1.096 


-.243 


1.393 


-.253 




4 


51 


.246 


-.095 


.335 


-.134 




5 


76 


.855 


-.098 


4.483 


-.221 




6 


68 


.808 


-.096 


.433 


-.068 



The entries in this ^olumn are F-statistics, results of testing the 
significance of for COP. 

★ 

The entries in this column are the standardized regression coefficients 
for the COP variable. 



2 . 2 Student Qpinionnaire Instrument 

Although generalized measures of attitudes are quite 
common, few attitude instruments appropriate for use in group 
testing of young children are generally available. Since a 
group administered test of affective impact of COP on students 
was desirable, we decided to develop an instriament for this 
purpose, ■ 

We could not adapt the typical adult attitude scale 
for several reasons. First and most important, these require 
a degree of verbal skill that cannot be expected of young 
children. Even if the standard instruments were read aloud 
to th^ students , they would not likely understand some of 
the words, and changes in the wording would alter the actual 
content of the scale. in addition, the response modes are 
complex. Rankings are clearly .too difficult. Agree/disagree 
responses introduce an acquiescence effect. It is not un- 
reasonable to assume that children tend to acquiescence and 
will therefore agree more than disagree, regardless of the 
content of the items. Further, commercially available scales 
are generally "trait measures" using a large number of items 
to measure a single underlying trait. Since we were interested 
in studying attitudes not as traits but as opinions, such 
scales were inappropriate. 

If the items are designed to be administered by being 
read aloud and the language is intended to be understandable 
to young children, the initial objections can be overcome. 
Market researchers have used a type of Likert scale with 
children with some success. In it, the options on the scale 
are represented by a series of faces with expressions changing 
from smiling to frowning. It has been demonstrated that chil- 
dren are able to respond appropriately in this mode. Finally, 
savings in time can be made by using the opinionnaire method 
rather than the trait measure approach. For e^tample, if we 
wish to know a student's attitude towards his teacher, we ask 
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him to mark the face most like his feelings toward the teacher. 
Marking a smile implies a positive attitude, marking a frown 
implies a negative one. since this instrument will not be used 
for individual diagnostic purposes, such general measures have 
sufficient reliability if group rather than individual data 
is used. Following these techniques we developed an opinionnaire 
appropriate for use with children in grades i-vi. It required 
no reading on the part of the students, since it was administered 
verbally. 

In order to explore differences in attitudes of students 
with/without COP aides, a test was developed which elicited re- 
sponses on a pleasure/happiness scale for 15 different stimulus 
words. We reasoned that response to anchor words like "ice 
cream" should not change between classes with/without COP aides, 
but that responses to words like "teacher" might. We knew from 
pilot testing that there were differences in response as a fun- 
ction of grade. Therefore, while this factor was not of impor- 
tance to this study it was included as a control variable. 

•All students for whom we had achievement test data were 
given the opinionnaire instrument and a 3x6 analysis of variance 
was performed on the results. To do this a separate analysis of 
variance was performed for each stimulus word with three levels 
of aide (COP aide, no aide, other aide) and six levels of grade 
(1-6). Table XII describes these, results. On first looking at 
these results we were encouraged because our hypothesis seemed 
to hold up. That is, none of the anchor words (ice cream, mail- 
man, going to bed, liver) had significant F's for the aide 
main effect whereas eight of the eleven critical stimuli showed 
significant F's for this aide main effect. 

An inspection of the means of these significant stimuli, 
however, showed that there were no trends to these results. 
Thus, in some cases, the direction of the difference favored the 
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1 

COP aided classes; in some it was reversed. 

Therefore, we must conclude that the results on the .1 
Studen t Opinionnaire Instrument do not show significantly better 
attitudes among child ren in classes with COP aides than among ~ 
those classes without COP aides. 
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TABLE XII 

ANALYSIS OF VARIANCE FOR ITEMS 
OF THE STUDENT OPINIONNAIRE INSTRUMENT* 



SCALE 


AIDE 




INTERACTION 


Ice Cream 


0.602 




0 . 710 


Mailman 


1 .958 


c C.cil if if if 


2 . 084** 


Going to Bed 


2 . 043 


in "^Q Ait it it 


4.671*'** 


Plavina f^pmpc; 




0 . 340 


3 .623*** 


Yourself 


6.503*** 


4.651*** 


1.475 


Other Children 


12.904*** 


11.935**^' 


3.060*** 


Stay Home 


131.843*** 


366.906*** 


38.052*** 


Teacher 


10.558*** 


26.218*** 


3.493*** 


Standing in Front 








of Class 


2.392 


9.386*** 


38.685*** 


Spelling * 


28.733*** 


9.141*** 


3.433*** 


Teacher Home 


0.895 


30.421*** 


4.451*** 


Liver 


1.371 


13.314*** 


14.836*** 


Mirror 


0.274 


8.225*** 


1.964** 


New Child 


22.778*** 


10.725*** 


1.302 


School 


6.571*** 


15.916*** 


0.972 



* Table entries are F-ratios for each main effect. 
** P < .05 
*** P< .01 
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3 • 0 Teachers 

Our research plan had anticipated comparing two groups 
of teachers; those who had COP aides and those who had no aides. 
The five sites who had been selected to provide us- with these 
teachers were unable to meet their assigned quotas. In consul- 
tation with the Office of Education, the design was modified to 
include teachers with aides other than COP. The resultant re- 
duced sample consisted of: 

Group I 30 Teachers with COP aides 

Group II 24 Teachers without aides 

Group III 13 Teachers with aides other than COP 

Data on this third group of 13 teachers was aggregated and is in- 
cluded in the cross-tab run in Appendix Volume II, pp, 143-210 (See; 
"Aide Type - Other"). However, we are restricting our present 
discussion to Groups I and II. This decision is appropriate not 
only because of the very small size of Group III (13) but also be- 
cause their aides' time in the class was distributed differently 
from the COP aides; that is. 

Time teacher has COP Aided Other Aide 

aide in class Teacher Teacher 

(30) (13) 

% % 

Full Time 10 39 

50 to 99% 53 15 

Less than 50% 33 46 

NA 4 0 

To control for confounding of type of aide and time in class- 
room would require a further breakout of cells with bases lower 
than 5 in several cases. 

The discussion which follows, then, describes responses of 
teachers with COP aides and teachers with no aides. 
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In assessing impact on teachers of having a COP aide 
we were interested primarily in two effects. First, we wanted 
to see whether a teacher with a COP aide could devote more of 
her time to instructional duties than a teacher who had no aide. 
Secondly, on the assumption that the COP aide might sensitize the 
teacher to the needs of her children we wished to compare the 
number of extra-classroom school related activities reported. 

As to the first, teachers with COP aides report that 
58% of their time is spent on instruction; 45% of the non-aided 
teacher's time is so spent. While this difference is in the 
expected direction, t-tests on the difference between means show 
them to be non-significant. The same holds for extra class- 
room activity. The list below shows the expected trend, but 
t-tests did not establish significance. 



Average number of field trips taken 
with class during last school year 

-Average number of parent-teacher 
conferences att.tmded during last 
school .year 

Average number of visits to stu- 
dents' homes made during last 
school year 

Teachers with COP aides were generally satisfied with 
COP (90%) with 57% giving it the highest rating of "very satis- 
fying, " 

When asked to rank success of goals of COP, teachers 
responded differently from both principals and superintendents. 
Table XIII shows these ranki/iys. Teachers with' COP aides were 
the only respondents whose rnean ranking was, "to raise achievement 
levels among low-income children," (All other groups ranked 



Teachers with 
COP aides 
(30) 

3,2 
21.3 
7,8 



Teachers 
without Aides 
(15) 

1,7 
18,3 
3,2 



first, "to attract low income people to a career in education") - 
On the evaluators' visits to the sites, teachers v/e spoke to 
were convinced that their students were doing better work and 
learning more by having an aide in the classroom- Most teachers 
want an aide in their classroom. Of those we sampled, 97% who 
had a COP aide stated they preferred a teacher aide in the room. 
Even those who did not have an aide wanted one; 83% preferred 
a teacher aide 'n the room. 

In the light of the results of the achievement test 
scores for students with and without COP aides and, beyond this 
study, the general negative results when one attempts to assess 
the impact of educational programs on comparable groups of 
children one is drawn towards the conclusion that either the 
teachers are incorrect in their perception or the tests are in- 
sensitive to achievement changes in the targeted children* s group. 

We are dealing here with a number of issues being 
vigorously debated in educational circles tdday^ It is recognized 
that achievement tests tend to discriminate best in the middle 
ranges. They have poor discr iminability at the low end of the per- 
formance continuum. COP aides are most frequently placed in 
classes where the learning needs of the children are greatest, 
that is, children who fall at the lower end of the achievement 
continuum. To use an instrument which is insensitive to change 
within this range prejudices the results. 

The challenge of an impact evaluation may well be to 
explore with teachers what they mean when they say that their 
children are doing better work as a result of COP aides (or any 
other program) and to develop, based on these explorations, 
reality based criterion of improvement in achievement/performance 
of these targeted children. 



TABLE XIII 



I-IEAN SCALE VALUES OF "SUCCESS OF GOALS OF COP'' 
AMONG SELECTED RESPONDENTS 



GOALS RANKED 



Sixteen Sites 



Three Large sites 



PRINCIPALS SUPERINTENDENTS TEACHERS^ PRINCIPALS SUPERINTENDENTS 
(179) (20)** (30) (167) 



(3) 



To attract low- 
income people to 
a career in 

education 2.62 

To encourage 
parents and com- 
munity partici- 
pation 4.11 

To improve 
teacher training 
through a work- 
study program 2.63 

To raise achieve- 
ment levels among 
low- income 

children 2.68 

To improve the 
efficiency of 
professional 

staff 2.88 



1,70 



3.30 



3.10 



3.15 



3.75 



2.80 



3.84 



2.56 



2.48 



3.32 



2.01 



3.37 



3.36 



2.77 



3,24 



1.33 



1,67 



3,00 



4.67 



4.33 



* QUESTION : Five goals of COP are listed below. Please tell us how well you think 
these goals are being met in your school system. Rank them on a scale from 1 (most 
successful) to 5 (least successful). (ENTER A '1' TO INDICATE WHICH OF THESE GOALS HAS 
BEEN MOST SUCCESSFULLY MET; ENT}^R A ' 2 ' TO INDICATE THE GOALS NEXT MOST SUCCESSFULLY 
MET AND CONTINUE TO ' 5 ' TO INTRICATE THE GOALS LEAST SUCCESSFULLY MET.) 

** Accounts for two sites with multiple Superintendents. 
+ With COP aides. 
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4 . 0 Principals 



Principals of elementary schools which used COP aides 
were surveyed by means of a self-administered^ mail back question- 
naire, A response rate of over 95% was realized; 179 principals 
returned their questionnaires. 

The majority of principals found the experience of COP 
aides in their schools "very satisfying" (60%) or "somewhat 
satisfying" (26%). When asked to indicate changes they would 
like to see instituted regarding COP aides, the most frequently 
cited was the desire to have "more aides in regular classrooms" 
(78%) . 

The principals perceive the major impact of COP aides 
in the increase in individualized instruction of children. 
Thus, 

& 95% see their classroom teachers as having more 
time to work with individual children; 

• 87% consider increased tutoring of individual chil- 
dren as an impact of the program on structuring 

of the school; 

• 59% would like to see more COP aides to work with 
" special" children. 

Do principals perceive COP aides as different from other 
teacher aides? Of the principals surveyed, 63% had had aides 
in their school before COP, that is 113 principals had experi- 
ence V7ith both COP aides and other teacher aides. Of these, 
62 or 55% indicated that they saw differences between COP aides 
and other teacher aides. 

When asked to specify what the differences were, the 
comments fell into the following categories: 

Positive conunents : 8 6% 
Negative comments: 14% 



The positive comments referred primarily to the aides' 
professionalism and dedication. (e.g., "...better understanding 
of the role of teaching," "all very dedicated," "better qualified, 
"more professional attitude," "much more reliable," "higher de- 
gree of preparation and proficiency," etc.) Negative comments 
showed no trends suggesting that the comments may be idiosyncra- 
tic to a single experience with aides (e.g., "COP aides have 
harder time relating to kids," or "need more direction --occa- 
sionally personal problems interfere with school work, also 
scheduling conflicts.") 
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Superintendents (LEA* s ) 



The survey of superintendents was intended- to explore 
changes in the school system that might be attributable to the 
presence of COP and also to demonstrate attitudes of the school 
system towards COP, Although 16 individual sites were surveyed, 
two of them (Denver, Colorado and Belcourt, North Dakota) operate 
in several distinct school districts; Denver in two; Belcourt in 
four. Therefore^ the data were aggregated three ays; for 14 sites 
havina a single rchool dis-^rict; for the two sites with six 
superintendents and for the sixteen sites combined. Table IV 
presents sele^u^d data from responses to the superintendents 
questionnaire for the. sixteen sites combined. By studying 
Table XIV, several observations can be n)ade. 

• The superintendents show high regard for COP aides. 

Fourteen of them consider the experience of COP aides 

in the school system "very satisfying" while only one 

out of twenty viewed it as "somewhat unsatisfying." 

★ 

Furthermore, most of the superintendents wanted more 
COP aides in regular classes (19) , more COP aides to 
work with special children (16) , and more COP aides 
to work with minority children (13) . 

• Since COP entered the school system, 17 superintendents 
noted the existence of increased linkages between 

the school system and community groups; half of them 
(10) noted increased access to federal funds . 

• Changes in hiring practices show a mixed picture with 
no predominating trends, 

• When asked to rank how xvell five listed goals of COP 
are being met, the highest* mean rank was "to attract 
low-income people to a career in education." 



* Total =20 
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TABLE XIV 

SELECTED MEASURES OF SUPERINTENDENTS OF THE SIXTEEN SITES COMBINED 

(Total number of superintendents = 20) 



TOTAL SUPERINTENDENTS 
(20) 

Experience in having COP aide s 

Vary satisfying 16 

Somewhat satisfying 2 

Neither satisfying nor unsatisfying 1 

Somewhat unsatisfying 1 
Very unsatisfying , .0 

Would like to see following changes 

More COP aides in regular class 19 

Fewer COP aides in regular class 1 

More supervision of COP aides by training staff 10 

More aides to work with special children 16 

More aides to work with minority children 13 

Changes in System since COP 

Increased linkages between school system and 17 
community groups 

Increased access to federal funds 12 

Increased discussions with local teachers unions 6 

No longer Now re- 
required/ quired/ 

Changes in hiring practices in past allowed allowed 

two years ' 

National Teachers Exam 5 2 

Full state certification 0 9 

Personal interviews 1 8 

Hiring of aides/teachers who have not 4 4 
met state certification requirements 

Partial/temporary certification 4 5 

No changes 7 
No answers 1 



(Table XIV cont'd) 

Attribute above Changes to COP 
Yes 6 
No 5 



- i^r . ^-oi ' .ir r n success of meeting COP goals Mean 

Rank 

To attract low-income people to a 
career in education 
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1.55 



To improve teacher training 3.00 
through a work-study program 

To encourage parent and community 3.10 
participation 

To 'raise achievement levels 3.10 
among low-income children 

To improve the efficiency of the 3.50 
professional staff 
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Col_'-.;j2e _Ad£r. strator .- (IHE ) 



To assess the impact of coP on institutions of higher 
education, Abt Associates conducted interviews with re . • ^sentatives 
from all of the colleges and j vp, sities who were a.v-r:' courses 
to COP aides in the .-.:Kteen s -.p^ed sites. Tr.-re K=re -otal of 
23 insti tur.;.ont- 'ne interview was conducted by means of a 
closed-ended questionnaire. Respondents were deans, other admini- 
strative officers and on occasion teachers of COP aides. They 
were drawn from a list submitted to us b:- rhe indiv:.aacn Project 
Directors. m collating the resi.ilcs each institution wa^ counted 
once . 

The impact of COP on institutions of higher education must 
be assessed within the limitations of the short time period that 
the COP program has been in existence. All of the 23 institu- 
tions made adjustments and accommodations n or 'er ^ o include 
COP students in their pro^:^- ,. These ranged r;.on ^ -rple course 
schedul- vnv ch- '.jes to radical changes in admissions policies. 

The kinds of course changes made in order xd accommodate 
COP students are described in Table XV below. 
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TABLE XV 



COURSB/F.COI.T. .CCOHMODATIOHS ^ COP ^f^f ^ lTrf^,7 

THE LAST TWO YEA.-:S (Among 23 Institutions or niyi 



Courses 

Content Changes 
simplified 

more related to work experience 
more minority group content 
generally reflective of "needs" 
of COP 

Meetj nq Time Changes 

Location Changes 

(i.e., off campus) 



Faculty 

H i r e d specifically f or COP 

Specialists in paraprof essional 5 
training 

Personnel from local school system 
Representatives of minority groups 
From low-income groups 



19 



8 
10 
3 
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As can be seen, all the schools changed their course 
content . The most frequent change was a shift from less theoreti- 
cal to more work-related content. The kind of work experience 
most frequently cited by the administrators was field work and 
practicum. 

More than half the institutions changed course meeting 
to accomodate COP; six gave courses off campus. 

What is unique about the above is that these courses were 
full credit courses. This is attested to t>y the fact that 17 
of the institutions reported that courses, designed for COP, were 
usually open to non-COP students. 

The flexibility of these institutions is further docu- 
mented by the following: 

• 14 colleges reported hiring faculty especially for 
the COP program. Of these, 10 stated that they, 
had hired representatives of minority groups. 

• 12 colleges stated that changes in admissions re- 
quirements had been made in the past two years. 

• 16 stated that changes in degree requirements had 
been made in the past two years. As detailed in 
Table XVI below, these changes were all in the 
direction of liberalizing the regulations. 

While the data describe changes already made, tha short- 
ness of the program makes the full impact of COP on these in- 
stitutions difficult to assess. The COP presence on campus, 
the unique student group it represents, its academic success 
or failure must surely have an effect on faculty and administra- 
tion attitudes. That this is indeed the case is demonstrated by the 
following sampling of direct quotes from the IHE questionnaire. 
These comments were made by college faculty and administrators 
and represent half of the sixteen sites: 
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" The COP program has served as a catalyst for change 
or at least re-evaluation throughout the; college. 
We have experienced great success in COP and this caused 
us to look more closely at requirements for all children. 

"We are on the threshhold of additional significant 
changes . " 

"The new program for regular students reflects lessons 
we've learned from working with COP and co-op students. 
The emphasis in the new requirements is on much more 
field experience." 

"The program has had a beautiful effect on changing fa- 
culty in Arts and Sciences and in Education. The coopera- 
tion and understandings that have developed; the team 
teaching and individualizing of instruction areas _ in 
liberal arts is something that is unique. We believe that 
this will help bring about change within the university.-' 

"Instituted the junior year internship as a direct result 
of COP. . . " 

"Working in the program is a real pleasure and at times a 
challenge. What some students may possibly lack in basic 
skills, they compensate for in energy, enthusiasm and mo- 
tivation. Generally, they do as well or better than 
regular college students." 

"The COP and ACTION programs have shown the faculty that 
students can succeed in a variety of program formats m 
the college setting; that field centered programs have a 
great deal of merit." 

"There will be some invaluable spin-off as we consider cur- 
riculum reform and experiment with state requirements _ for 
teacher certification going from approved program to in- 
dividual competency. . .It has been an inspiration for many 
disadvantaged students who want to get into the program for 
learning's sake and vocational self-improvement instead 
of dollar subsistence." 

Thus, in only two years, the COP program has effected 
changes in institutions of higher learning both in present status 
and future expectations. To this extent it has demonstrated a 
noticeable impact. 
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TABLE XVI 



SPECIFICATIONS OF CHANGES IN ADMISSION/DEGREE REQUIREMENTS IN 
INSTITUTIONS WHICH HAVE EXPERIENCED SUCH CHANGES IN THE PAST 
TWO YEARS (Among 23 Institutions of Higher Learning) 
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Institutions Reporting Changes 

in Admission Requirements 12 



Open admission for COP only 3 

Open admission for all 2 

Acceptance of GED 2 

Reduction of weight assigned standardized 

admission test (e.g., CBS) 3 

Waiver of Diploma, GED, other requirements after 2 
specified number of credits completed (i.e., 
student enrolled first as special student) 



Institutions Reporting Changes 

in Degree Requirements 16 

More flexibility/fewer required courses 6 

Reduced number of credits 1 

Reduced CPA requirement 2 

Credit given for experiences 2 

New certification program 2 

Changes not specified 3 
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State Education Agencies (SEA's): Survey Findings 



One implicit goal of COP is to effect changes in state 
teacher certification regulations to better accommodate the 
career preparations of paraprof essionals • In order to 
assess OOP's success in meeting this goal Abt Associates Inc. 
conducted a telephone survey of state certification officers in 
each SEA to determine the extent to which changes have occurred 
or are planned in each state and the extent to which the certi- 
fication officers see their changes as responding to the needs 
of COP participants. Interviews were complete with 51 of 52 
possible certification officers*, the exception being Massachu- 
setts whose officer, despite 10 attempts, could not be reached. 

When discussing changes, we asked the certification 
officers about a specific set of eight changes. We were inter- 
ested in whether changes in these areas had occurred in the 
past three years or were anticipated in the near future. These 
areas concerned changes such as: 

a. Granting of provisional certification 

b. B.A. no longer required 

c. Number and/or type of courses required 

d. Credit given for work experience outside of the 
teaching profession 

e. Credit given for hours of teaching beyond required 
practice teaching 

f. Exemptions made for particular Local Education 
Agencies (from state regulations) 

g . Granting of special certification for teachers ' aide 

h. Extensions in time requirenients for certification 

(In addition, the respondents were asked to specify any other 
changes which had been made or were planned.) 



* 

Includes Puerto Rico and District of Columbia 



WHAT KINDS OF CHANGES IN STATE CERTIFICATION RULES HAVE BEEN 
MADE OR ARE PLANNED? 

The number of positive responses to each of the eight 
changes listed above constitute an indicator of the amount' of 
change occurring in the SEA. (A positive response is one in 
which the specifics of the change described are consonant with 
the purposes of COP.) Table XVII summarizes these changes. 

The most frequently cited changes were as follows: 

• Granting of provisional certification 

• Number and/or type of courses required 

• Credit given for work experience outside of the 
teaching profession 



(See following page for Table XVII.) 
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TABLE XVII 





Changes Made or Planned in State Certification Rules 
As Cited By State Certification Officers 










Number 


of Stat^js 




A r ^1 Changes Made 
Area of Chang_e_ T,;.st Three Ycarc 

a. Granting of provisional , 23 
certification 


v^iictjipefa x^i.anneci'»* 
16 




b. 


B.A. no longer required 


4 




7 


- 


c. 


Number and/or type of courses 
required 


23 




23 




d. 


Credit gix-en for work experience 
outside of the teaching profession 


9 




•* 

16 


■ " 


e. 


Credit given for hours of teaching 
beyond required practice teaching 


2 




4 




f. 


Exemptions made for particular 
Local Education Agencies 


4 




5 




g- 


Granting of special certifi- 
cation for teachers^ aides 


6 




11 


i 


h. 


Extensions in time require- 
ments for certification 


5 




8 


[ 




Don^t Know 


0 




2 





- The total of 76 changes made in the last three years and 90 anticipated 
changes represent responses of 46 SEAs; 5 SEAs neither made nor plan 
changes. 



IS THERE ANY RELATIONSHIP BETWEEN PRESENCE OF COP AND 
CHANGES IN STATE CERTIFICATION RULES? 
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The scope of the present research permits us to correlate the pres- 
ence of COP and the number of changes.' Of course, while a simple corre- f 
lation of two variables is never sufficient to establish causality, the lack of 
a correlation indicates that tlxere is little possibility of any causal relation- 
ship. With this in mind, we proceeded to investigate the relationship of the 
presence of COP projects and the amount of change. 



i 
i 



Since most states have at least one COP project, we cannot compare 
states with projects to those without. However, it is possible to quantify the 
decree of presence of COP in a state in several ways. We have chosen to look 
at the number of proiectj_and the number of participants in a state as our 
variables. The question we are asking is, "Have states with more projects 
and/or participants made or planned more changes?" 

The table below shows the correlation obtained between the number of 
sites in a state/number of participants in a state and the number of changes 
made/number of changes planned: 



Correlations of Changes Made/Planned 
And Number of Sites /Participants in a State 

Number of Changes Number of Changes 
^ade Planned 

(51) (49)^;= 

Number of Sites +.19 ^ 35 

Number of Participants +.26 ^ 53 



*Basereduced by 2 respondents who answered "Don't Know. 
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From the correlations in the preceding table we note: 

• All relationships are positive 

• Number of participants are more closely related to 
extent of change than number of sites, particularly 
m "planned changes." 

• COP is strongly associated with planned change. 

The latter finding reflects the fact that COP is a new program 
whose impact would be expected to take some time in extending up- 
ward. (An alternate interpretation would hold that the positive 
correlations reflect the fact that states that were interested 
in change were interested in COP.) 

It should be noted that the correlations presented are 
exact values rather than sample estimates. Since we have sampled 
51 out of 52 elements of the universe, we may assume that any ob- 
served differences are real differences rather than possible sam- 
pling errors. We also assume relatively error-free measurement, ' 
at least to the extent that the respondents were both knowledge- 
able and truthful. 
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In interpreting these results we must take into considera- 
tion possible causes for the covariation of COP and change that 
has been found. In the case of the strongest relationship (num- ' 
ber of participants and number of planned changes) , the variables 
have approximately 27% of their variance in common. 

However, one must be cautious in interpretation of results. 
Correlation does not mean causation; attribution does not follow 
from significant correlations. Rather, significant correlations 
suggest further lines of inquiry. The present study has shovm 
that there is some kind of relationship between the presence of 
COP projects in a state and the amount of change in the state's 
certification requirements. Further investigation is needed to 
establish if there is some causal relationship between the vari- 
ables or if the correlation is merely a spurious one, due to both 
variables covarying with some other variables not measured in 
the present study. 

Additi onal Data Analysis for SEA ; Attribution of Change to COP 

One possible strategy that can be used to investigate 
the direct impact of COP on the certification requirements is 
to ask persons in a position to know if they perceive COP as 
having any effect. Clearly, state certification officers are 
in such a position, and our interviewers asked them for their 
opinions on this subject. The exact questions asked were: 

o First Question : "In your opinion, will these future 

changes (in certification requirements) reflect a con- 
cern for the participants in the Career Opportunities 
Program, that is, COP?" 

• Second Question : "In your opinion, do your present 
teacher certification regulations reflect a concern 
for the participants in the Career Opportunities Pro- 
gram, that is, COP?" 
Shown below are the response of certification officers of SEAs 
to these questions: 

First Question Second Question 

(51) (51) 

Yes 21 20 

To 5 21 

TcLlTel Planned 
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The certification officers of two states, when asked 
the first question, replied that they had never heard of COP. 
As a result, they were not asked any further questions about 
the responsiveness of their state. It is interesting to note 
that one of the states (Kansas) has no COP projects, whereas 
the other (Louisiana) has four projects. 

An index of responsiveness to the needs of COP partici- 
pants was constructed from the responses to these two questions. 
A state which now perceives itself as fully responsive to COP 
would answer "no" to the first question (about planned changes) 
and a state which is not now responsive would answer "no" to 
the second (present requirements). However, states that are 
not presently responsive and have no plans to become so would 
answer "no" to both. This mixing of states where positive, changes 
have or will occur with states that are not responsive could 
cause some problems in the analysis of these results. Consequently, 
we have constructed a composite index of "responsiveness to COP" 
from the responses to the two questions. States that answered ■ 
"yes" to either question are considered "responsive." States that 
did not are considered "nonresponsive . " On this scale 31 states 
are "responsive" to the needs of COP and the remaining 20 are 
considered "nonresponsive." Point biserial correlations relating 
the presence of COP with "responsiveness" showed positive corre- 
lations of +.19 for number of sites; -h.26 for number of parti- 
cipants. 

We conclude therefore that not only is there a significant 
relationship between presence of COP in a state and the amount 
of change reported, but also that there is a relationship be- 
tween that presence and the extent to which these changes are 
seen as taking into account the needs of COP. 



Ill, CONCLUSIONS AND RECOMMENDATIONS 



While the problems of conducting an impact evaluation of 
an educational program are familiar to most readers, it is appro- 
priate to review them now lest we neglect to temper our conclu- 
sions by the reality constraints we labor under. We wish to make 
the following preliminary observations. 

• It is very difficult to assess impact of any rela- 
tively new program, Start-up time can be conserva- 
tively estimated at six months to one year. It takes 
at least that long to get a program running smoothly. 

• Teacher aide programs exist in most of the sites 
where COP is located. Therefore, it does not 
appear in the classroom as something totally unique. 
It is not a treatment that can be applied to a popu- 
lation that is "uncontaminated , " In other words, 

a non-treated control group is difficult to find. 
This assumes that an appropriate control group would 
be a non-teacher-aided one , 

• Exogenous factors can mask impact effects should 
they occur, 

• When the evaluator is forced to leave decisions and 
actions in the hands of non-research oriented local 
administrators, deterioration of field control and 
subsequent reliability of data is inevitable. 

These problems are ubiquitous to social science research; 
they were present in this study. We feel that the strategy of 
federal intervention to facilitate evaluative studies could bene- 
fit greatly from some hard study. 



The findings show that the progra.ni is success- 
ful when measured by the following impacts: 

• COP aides show strong motivation to continue in 
the program and become teachers. They have a posi- 
tive professional view of themselves. They are 
representative of the population originally targeted 
for this program. As such, the program has provided 
a vehicle for upward mobility for the aides. 

• Satisfaction with the program is high among . 
principals, superintendents, teachers, and COP 
aides . 

• Principals wanted more COP aides in their class- 
rooms and felt that a major impact the aides had was 
to increase the amount of individual instruction 
scheduled for children. They preceived COP aides as 
more professional — more serious than other teacher 
aides . 

• Superintendents saw the COP aides as increasing link- 
ages between the school system and community groups, 
(The COP aide frequently acts as an "interpreter" of 
neighborhood needs to the administration.) They wanted 
to have more aides for both regular classes and for 
special students. 

• Institutes of Higher Education reported changes in 
their course content, schedules, college requirements 
not only as an initial accommodation to COP in the 

first place, but also as a result of their COP experiences. 
These changes, present, planned, or being considered 
for all students were in the direction of performance 
based teacher education . 
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# state Education Agencies showed a positive relation- 
ship between presence of COP and amount of change 
reported . 

On the negative side is the lack of evidence for impact 
on student performance/attitudes. Furthermore, while teachers 
were enthusiastic, differences in their performance, while in 
the expected direction for each measure, were not significantly 
different from teachers who did not have COP aides. 

On balance, the program can be judged as successful 
in its impact on both COP aides and the participating IHE's. 
The opportunity for upward mobility and the eye-opening exper- 
ience which many academicians have had as a result of the COP 
presence argues well for the continuation of this program. 



APPENDIX 



THE THREE SITE SURVEY: FINDINGS 



Er|c 68 



THREE-SITE RESULTS 

The main COP impact evaluation was conducted in 16 pro- 
jects selected from a restricted set of COP projects funded in 
fiscal 1972. It was felt that extremely large sites (with more 
than 200 participants), located in urban areas, would bias the 
results because. of their size. Rather than assume that such 
sites are qualitatively the same as smaller sites, and that 
there was cause for combining them with the other sites, it was 
decided, in consultation with the client, to study three such 
sites and compare the findings of this substudy with those of 
the main study. That is, rather than assume them to be similar 
(or different), we treated the question as a testable hypothesis. 
The three sites selected by OPBE and the COP program officer were 

• Chicago, Illinois 

• Detroit, Michigan 

• Bedford-Stuyvesant (Brooklyn) , New York 

The survey effort at these sites was, as. agreed, a limited 
one. Students and classroom teachers were not involved as re- 
spondents. The COP participants themselves responded to the 
spring questionnaire only; they did not receive the fall instru- 
ment and were not tested with the ABLE. 

In addressing the COP participant data, it became evident 
that certain analyses could not be performed. Obviously, we 
could make no statements about the basic skills of these parti- 
cipants, since we had no measures of them. Neither could we 
discuss changes in their career aspirations over time, since we 
had only one set of data on the participants. Further, we could 
not adequately investigate sex differences, since there were 
too few males in the sample (only 23 males out of 212 participants 
in the three sites). 

When the data was sufficient to allow us to perform the 
analyses of interest, our results were similar to those of the 
16-site sample, with a few notable exceptions. 
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It will be possible to make some, but not all, of the 
same descriptive analyses lor the participants in the three sites 
as were made for the total sample. In such cases, a description 
of the three sites will be made and then compared with the 
16-site sample. 

WHO ARE THE COP AIDES ? 

Overall, in the three sites, we find that they tend 

to be : 

• From families with incomes less than $6000 (53%) 

• Drawn from ethnic minorities (94%; 81% Black and 
13% Spanish) 

• Female (89%) 

• Neither young nor old (50% are 35 or older; 27% are 
4 0 or over and 12% are younger than 25) 

• High school graduates (92%) 

• Previously teacher aides (81%) 

• Community residents (80%) 

The magnitude of difference between first and second year parti- 
cipants observed in the 16 sites is not as evident here. 
There are small differences, not more than one or two percentage 
points, and they are not statistically significant. 

A possible explanation for this lack of shift is that 
the supply of minority, low-income candidates in the urban areas 
served by these three projects was not depleted by the first 
year's recruiting. The project directors indicated on their 
questionnaire that, if recruitment practices changed at all, 
it was in the direction of being more selective; selective on the 
basis of motivation rather than qualifications (previous educa- 
tion, etc.), indicating that there is an abundant supply of 
possible participants. 

When the three sites' participants are compared with the 
participants of the other sites on the above variables, we may 
note some similarities and some marked differences. Responses 
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to family income (53% vs. 51% under $6000) and previous edu- 
cation (92% vs, 93% high school graduates) are virtually iden- 
tical. The size of the female majority is greater in the three 
sites (89% vs. 74%). There are, however, significantly more 
minorities (94% vs. 68%), community residents (80% vs. 53%) 
and previous teacher ' ai .es (81% vs. 46%). The ages of urban 
participants are also more variable than those in the 16 sites. 
It is quite likely that these differences simply reflect the 
composition of urban areas, the differences resulting from the 
fact that there were many non-urban projects included in the 16 
project sample. For example, the opportunity for employment as 
teacher aides (supported by Title I, etc.) is much greater in 
urban areas. 

WHAT ARE THE CAREER PLANS OF COP AIDES ? 

Aides in the three sites were asked about rhoir career 
plans for the next year and for the next three years. Their 
responses v/ere quite similar to those of the participants in 
the 16 sites. Overall, 88% of the participants plan to be in 
COP next year (as compared to 8 9% in the 16 sites) and 43% Plan 
to be in COP three years from now (as compared to 46% of those 
in the 16 sites). When this is broken down by year in program, 
57% of the first year participants and 36% of the second year 
participants plan to remain in COP until three years from now. 
This split is similar to that of the 16 sites. With respect to 
other plans for three years from now, 58% indicated they plan 
to become teachers then, v/hile only 6% plan to go out of educa- 
tion entirely. Surprisingly, less than 2% plan to return to 
positions as teacher aides. Many more participants in these 
sites indicate that they will use their training as intended 
by COP (that is, in the classroom) than participants in the 16 
sites . 



WHAT IS THE COP EXPERIENCE ? 

The participants in the three large urban sites have a 
very positive perception of COP; 93% are "satisfied" or "very 
satisfied" with the program (with "very satisfied" accounting 
for 59%) . This is quite similar to the response pattern of 
participants in the 16 sites, although somewhat less positive. 
• There were absolutely no differences between the responses of 
first and second year participants to this item. 

When asked v;hat aspect of the program they liked the most, 
participants in the three sites responded as follows: 

• Chance to get an education (60%) 

• Financial assistance (17%) 

• Project staff (12%) 

This pattern is similar to that of the participants in the 16 
sites. When asked v;hat they liked the least, participants in 
the three sites responded as follows: 

a No complaints (14%) 

• Content of program (12%) 

• Communication with staff (12%) 

in both of the above situations, no other item was marked by 
more than 10% of the participants. The pattern of dislikes 
among the participants in the 16 sites is slightly different, 
although "no complaints" is also the response most frequently 
given. 

Since the educational aspect of the program is very im- 
portant to most participants, we should look a bit more closely 
at it. We find that 

• Most (69%) of the courses are required 

• The Modal number of courses taken is 3 (37%) , with 
55% taking 3-5 courses and 73% in the 2-4 course 
range. (This indicates that participants in these 
sites have somewhat fewer courses than those in the 
16 sites) 
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• Most (91%) rate the course work "about right 
for me" 

• 68% spend fewer than 10 hours on college assignments 

HOW DO COP AIDES FEE T ^OUT THEMSELVES AS PROFESSIONALS ? 

COP participants were asked to rate themselves on spe- 
cified performances as compared with other aides. The following 
table presents the results of this. It can be noted that 
aides have a very positive self image, 

SELF-RATINGS OF COP AIDES AS COMPARED WITH OTHER AIDES 

(jj^212) (Percentages) 

. Percent rating self as; 

Better About same Not as good No response 

Presenting lessons 36 S3 3-8 

Getting ideas across 44 46 2 8 

Getting along with 43 51 1 5 

parents 

Understanding child's 52 43 . 1 4 

needs 

Ability to instruct 45 50 ' 0 * 5 

individually 

Ability to tutor 44 51 1 5 

Ability to teach in 51 44 1 4 
small groups 

Lesson planning 28 54 9 9 

College performance 24 65 3 8 



Participants were also asked if they ever experienced difficulties 
in certain areas. Responses to this item are contained in the 
following table: 



FREQUENCY OF FINDING ACTIVITY A PROBLEM (Percentages) 

Very Very No 

Activity freq, f reg, occas . Seldom Seldom Response 

Giving a class 

lesson 2 1 15 30 35 17 

Getting children 

to work with me 3 0 9 23 51 15 

Getting along 

with teacher 2 2 5 12 61 17 

Maintaining dis- 
cipline 3 7 26 23 30 11 

Getting children 

to Ixke me 3 1 3 13 63 17 



It appears that the COP participants in the three large 
sites are, in general, much like COP participants throughout 
the country (as represented by the sample of 16 sites). What 
differences do occur are likely the result of special conditions 
of urban areas rather than the result of some difference in 
the quality of the procram. 



